Concerning the accuracy of MAST E-values

A recent article on the IMPALA algorithm by Schäffer et al. (1999) gives the misleading impression that the E-values returned by the MAST algorithm (Bailey and Gribskov, 1998) are unreliable. This erroneous conclusion was reached due to an inappropriate protocol in the experiment involving MAST in the IMPALA article. We have tested MAST using a redesigned protocol that causes MAST to behave as the IMPALA authors intended. This test reveals that MAST E-values are just as accurate as those reported by two of the other search algorithms studied in the IMPALA article, and nearly as accurate as those reported by the IMPALA algorithm. Superficially, MAST and IMPALA appear to be designed to perform complementary operations, but they are not. MAST is designed to search a target database of protein or nucleotide sequences for matches to a query consisting of a set of position specific scoring matrices (PSSMs). MAST treats the set of PSSMs as a single signature describing, for example, a protein family or regulatory region. Consequently, for each sequence in the database, MAST computes a single similarity score that combines the similarity of the sequence to each of the PSSMs in the query set. On the other hand, the IMPALA algorithm is designed to search a database of PSSMs for matches to a protein or nucleotide query sequence. Unlike MAST, however, each PSSM is treated as being the signature of a separate family, and scores from different PSSMs are never combined by IMPALA. Thus, using MAST to search a database consisting of a single sequence using a set of PSSMs as the query has an entirely different effect than using IMPALA to search the set of PSSMs using the sequence as the query. The experimental protocol used in the IMPALA article does not run MAST in the way the article’s authors intended, and virtually guarantees that the E-values will not be accurate. To measure E-value accuracy, the article’s protocol runs 467 MAST searches. In each case, the query is the entire set of wolf1187 PSSMs (converted to MAST input format), and the target is a single sequence. Since all of the PSSMs were placed into a single file, MAST treated them as the signature of a single protein family, rather